The subject matter of the present disclosure relates to correction of radial distortion in image data. The problem of radial distortion is common and well-understood in the field of image acquisition. The most common cause of radial distortion is irregular curvature of the imaging element, which is most commonly a lens. Radial distortion is manifested as either a shortening or lengthening of the radial distance in the image plane from the principal point of the image. Accordingly, the image is said to have suffered either barrel or pincushion distortion. The effects of barrel and pincushion distortion upon a rectilinear grid are shown in FIG. 1.
Radial distortion is a function of the lens material, construction, and the like, and is generally not present in expensive high-end consumer cameras because such cameras tend to use high-quality lenses such as rectilinear lenses. However, other less-specialized devices, such as mobile phones that include cameras, employ simple convex plastic lenses that induce considerable radial distortion. In addition, the drive toward smaller form factors, for example to make mobile phones thinner, can result in a very small lens where radial distortion is accentuated.
Conventional cameras correct for radial distortion in a variety of ways. High-end cameras incorporate expensive optical elements such as rectilinear lenses. Consumer cameras such as digital cameras employ a late (post-acquisition) correction of the image to account for the distortion. FIG. 2 shows a simplified block diagram of a conventional digital camera 200.
Referring to FIG. 2, camera 200 includes a lens 204, a sensor 205, a color synthesis module 206, a zoom module 208, a memory 210, a distortion correction module 212, a preview screen 214, a compression module 216, a pre-synthesis module 218, and a post-synthesis module 220. Lens 204 focuses an image upon sensor 205, which provides raw image data (also referred to as mosaic image data) to color synthesis module 206. In the mosaic image data, each pixel represents only one color. For example, according to the Bayer mosaic pattern, each pixel represents red, green, or blue.
Pre-synthesis module 218 performs functions upon the mosaic image data such as dead pixel correction, black level clamping, linearization, vignetting correction, channel gain for white balance, and the like. Color synthesis module 206 demosaics the mosaic image data, thereby producing color-synthesized image data (also referred to as demosaiced image data). In demosaiced image data, each pixel location represents all of the colors of the mosaic. For example, each pixel can comprise a RGB triad where each of the red, green, and blue channel components are available. Post-synthesis module 220 performs functions upon the demosaiced image data such as color correction, gamma correction, and the like.
Zoom module 208 scales the demosaiced image data according to a scaling factor, which can be provided according to operation of a wide-angle/telephoto control button of camera 200. Zoom module 208 stores the scaled demosaiced image data in memory 210.
Distortion correction module 212 retrieves the image data from memory 210, corrects the image data for the radial distortion introduced by lens 204, and stores the corrected image data in memory 210. The corrected image data can then be provided to preview screen 214, compressed by compression module 216 according to a compression standard such as the Joint Photographic Experts Group (JPEG) standard, and the like. The image data may also be provided for video compression using MPEG/H.26x compression standards, or may also be provided to an analysis module for evaluation of scene structure/detail to be used for face tracking, smile detection, or other applications of machine vision.
Conventional distortion correction modules 212 typically employ a geometric transformation of a two-dimensional object in a three-dimensional space. The transformation can be approximated to a polynomial relationship between the distorted and undistorted image points. Most of these polynomials assume the distortion function to be a pure, odd function of the radius from the principal point of the image. In order to account for other anomalies in lens construction, distortion correction module 212 may use a look-up table to derive any non-uniform stretching required on the image. This correction operation is computationally intensive and is therefore performed in software or by dedicated graphics processing engines.
These conventional solutions pose a number of practical problems. During the correction some pixels are mapped outside the viewing pane, and are therefore omitted, leaving unmapped pixel places (also referred to as holes) in the viewing pane. To alleviate this problem, a smoothing algorithm is usually executed on the transformed image, thereby increasing the number of operations, which in turn increases the band-width to memory 210. These practical challenges may prevent correcting radial distortion at run time or during the preview of the image.